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DETAILED ACTION 
Response to Arguments and Amendments 

Applicant's amendment filed May 20, 2005, has been entered and made of 
record. 

Regarding the objections to claims 2 and 16, in view of applicant's amendments 
and arguments, the claim objections are withdrawn. 

Applicant's arguments filed May 20, 2005, have been fully considered but they 
are not persuasive. 

As to the 35 U.S.C. 102 rejections, the applicant argues that the examiner's 
statement that "'groups' are considered to be the same as 'states'" is incorrect. The 
examiner disagrees and points the applicant to the applicant's own summary in the 
instant application (page 2, lines 31-33) where the applicant discloses that states may 
be modeled through any number of techniques including clustering. The dictionary 
definition of 'clustering' is "a grouping of a number of similar things" (see 
www.dictionarv.com) . 

As to the 35 U.S.C. rejections of claims 7 and 9, the applicant argues that there 
is no motivation for combining Yoshida (U.S. Patent 6,317,160) with Endo (U.S. Patent 
Publication 2002/0039203) and that it would require changing the principle of operation 
of Endo (the primary reference). 

In response to applicant's argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed Invention 
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where there is some teaching, suggestion, or motivation to do so found either in the 
references themselves or in the knowledge generally available to one.of ordinary skill in 
the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and In re 
Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, both references 
deal with obtaining images from a plurality of cameras. As has been previously stated 
by the examiner, combining Yoshida and Endo would enable the application of image 
processing techniques in Endo to a greater variety of camera systems, thus providing 
motivation. 

In response to applicant's argument that Yoshida could not be combined with 
Endo without changing the principle of operation of Endo, the test for obviousness is not 
whether the features of a secondary reference may be bodily incorporated into the 
structure of the primary reference; nor is it that the claimed invention must be expressly 
suggested in any one or all of the references. Rather, the test is what the combined 
teachings of the references would have suggested to those of ordinary skill in the art. 
See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). 

Although, the rejections mentioned above are still valid the examiner has found 
additional prior art that was not cited in the previous office action, which will now be 
used to additionally reject the claims. 

Applicant's arguments with respect to claims 5, 10, 12 and 14, have been 
considered but are moot in view of the new ground(s) of rejection. The examiner 
acknowledges that the claims were rejected using patents and publications assigned to 
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the instant application's assignee. The examiner is therefore using new references to 
reject the claims. 



Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 



Claims 1-6 and 11-16 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Foote (U.S. Patent 6,751,354). 

As to claim 1 , Foote discloses matching an input image with one of a plurality of 
states (Figure 2), the input image determined from a signal comprising images from a 
plurality of cameras (column 13, line 1-4); and assigning the input image to one of the 
states when the input image matches the one state (column 7, lines 23-24). Here the 
classes are considered to be the same as states. 

As to claim 2, Foote discloses determining an input image from the signal 
comprising images from a plurality of cameras (Figure 2 and column 13. line 1-4); and 
determining input image information from the input image (Figure 2, element 208 and 



Application/Control Number: 10/034,670 Page 5 

Art Unit: 2623 

column 6, lines 64-65 - here video features are a form of image infomnation); the step of 
matching further comprises the step of comparing the input image information with state 
image information corresponding to each of at least one states (column 7, lines 14-15); 
and the step of assigning further comprises the step of assigning the input image to one 
of the states when the input image information matches state image infomnation of the 
one state (column 7, lines 19-21). 

As to claim 5, Foote discloses there being a plurality of states (column 7, lines 2- 
4) and the step of performing training to determine the plurality of states (column 2, lines 
40-41). 

As to claim 6, Foote discloses outputting the input image, the input image output 
as being associated with Its state (Figures 30 and 34 and Column 23, lines 26-28 and 
Column 35, lines 41-46). 

As to claim 1 1 , Foote discloses that the input image information matches state 
image information of the one state when a metric comparing the Input image information 
and the state image infomnation of the one state falls within a predetermined value 
(column 23, lines 20-21). In this case feature vectors are being compared with 
infomiation gathered from training images and a threshold is used for comparing. A 
threshold must be set before something is compared to it, meaning that it must be 
predetermined. 

As to claim 12, Foote discloses the step of determining input image information 
from the input image comprises the step of determining a histogram from the input 
image (column 26 lines 47-49); the step of comparing comprises the step of comparing 
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the histogram of the input image with histograms corresponding to each of at least one 
states; and the step of assigning comprises the step of assigning the input image to one 
of the states when the histogram of the input image matches the histogram of the one 
state within a predetermined error (Column 26, lines 49-52. In addition see claim 1 1 for 
discussion on predetermined thresholds/error etc.). 

As to claim 13, the limitations of the claim are rejected for the same reasons as 
givien in the rejection of claim 2. Note that the Foote discloses features, plural, implying 
a plurality of features. Predetermined threshold/error is discussed in the claim 1 1 
rejection. 

As to claim 14, Foote discloses states that are states of a Hidden Markov Model 
(column 7, lines 16-18). 

As to claims 15 and 16, Foote discloses the use of memory and a computer 
system (Figure 1) for carrying out image classification methods. The use of software is 
inherent on a computer implementing a method. The other limitations are rejected for 
the same reasons as in claims 1 and 2. 

Claims 1, 2, 6, 11, 13, 15 and 16 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Endo (U.S. Patent Application 2002/0039203 A1). 

As to claim 1 , Endo discloses matching an input image with one of a plurality of 
states (paragraph 0117, lines 5-7). Here the image is classified into a group depending 
on which physical location the image was originally taken in. There is a plurality of 
groups (paragraph 0099, line 4 mentions "groups" as opposed to "group") and "groups" 
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are considered to be the same as "states". The above sections of Endo describe an 
invention that is present on a digital camera. Endo does however state that it can be 
applied to a personal computer, which is not part of a camera (paragraph 0151, lines 7- 
10). Endo also discloses that when in this form, the input image is from a signal 
(paragraph 0158, lines 3-4, if electronically transmitting, it must be a signal) with images 
from a plurality of digital cameras (paragraph 0158, lines 9-10). Endo discloses 
assigning an input image to a group if that image matches the requirements of 
belonging to that group. This is disclosed by the use of thresholds to determine which 
group an image belongs to (paragraph 0099, lines 1-4). 

As to claim 2, Endo discloses determining input information from the input image 
(paragraph 01 17, lines 5-7). Here the images are classified into groups based on 
position information (a form of input information), after that information has been stored 
with each image (paragraph 01 17 lines 2-3). Endo also discloses comparing input 
image information with state image information corresponding to each state (paragraph 
0099 lines 1-4). Here it shows that each group only accepts images from certain 
locations. This means that location information from the image must be compared to 
locations information of the group. Endo discloses classifying (or assigning) the input 
images to a group when the input image information matches the group image 
information of that group (paragraph 0099, lines 1-4, see discussion above). 

As to claim 6, Endo discloses outputting an input image in a manner that shows 
what group it belongs to (paragraph 0159, lines 3-4). 
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As to claim 1 1 , Endo discloses the use of a predetermined ("fixed") threshold for 
detemnining if an image belongs to a group (paragraph 0099, lines 1-4). 

As to claim 13. Endo discloses determining a plurality of features from the input 
image (Figure 7 - here several types of locations information are associated with each 
image). Since the image is compared to other groups in terms of location (see claim 2), 
it follows that a plurality of information is being compared between the input image and 
the groups (i.e. latitude, longitude etc.). Regarding the limitations about assigning the 
image to a state and using thresholds, the limitations are rejected for the same reasons 
as in claims 2 and 1 respectively. 

As to claim 15, Endo discloses using a personal computer (paragraph 0157, line 
2), which inherently must have a processor and memory with readable code for it to 
work. The other limitations are rejected for the same reasons as in claims 1 and 2. 

As to claim 16, Endo discloses using a personal computer to implement steps of 
the method (paragraph 0157, line 2). The use of software is inherent on a computer 
implementing a method. The other limitations are rejected for the same reasons as in 
claims 1 and 2. 

Claim Rejections - 35 USC § 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sk\\\ In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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Claims 7-9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Foote in view of Yoshida (U.S. Patent 6,317,160). 

As to claim 7, Foote as applied above does not expressly disclose that the 
images on the input signal are asynchronous. Yoshida discloses images on input 
signals being asynchronous (column 4, line 24). 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use the image processing techniques in Foote to process the 
asynchronous images in Yoshida, as they both deal with images obtained from a 
plurality of cameras. 

One would have been motivated to do this, as it would enable the methods to be 
applied to a greater variety of camera systems. 

As to claim 8, Foote does not expressly disclose multiplexing images onto a 
signal where the sequence of cameras from which the images are coming is not 
predetermined. Yoshida discloses multiplexing images onto a signal (Figure 1,11) and 
further discloses that these images are in an asynchronous state (Column 6, lines 66- 
67) and does not state that the sequence of cameras is known. In addition to this, an 
identification signal is used to identify which camera an image came from (column 3, 
lines 13-16). These pieces of information together, further imply, that the sequence of 
cameras is unknown. 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use the image processing techniques in Foote to process 
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images tliat are input from a signal containing images from a plurality of cameras, of 
unknown sequence, as in Yoshida, as they both deal with images obtained from a 
plurality of cameras. 

One would have been motivated to do this, as it would enable the methods to be 
applied to system where the camera sequence is unknown, hence resulting in 
applicability to a greater number of systems. 

As to claim 9, Foote as applied above does not expressly disclose that the 
signals on the input signal are synchronous. Yoshida discloses images on input signals 
being synchronous (column 6, line 67 to column 7, line 2). 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use the Image processing techniques in Foote to process the 
synchronous images in Yoshida, as they both deal with images obtained from a plurality 
of cameras. 

One would have been motivated to do this, as it would make processing easier 
as more is known about a synchronous image signal prior to processing than an 
asynchronous signal. 

Claims 7-9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Endo 
as applied to claims 1 and 2 above, and further in view of Yoshida (U.S. Patent 
6,317,160). 



Application/Control Number: 10/034.670 Page 1 1 

Art Unit: 2623 

As to claim 7, Endo as applied above does not expressly disclose that the 
images on the input signal are asynchronous. Yoshida discloses images on input 
signals being asynchronous (column 4, line 24). 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use the image processing techniques in Endo to process the 
asynchronous images in Yoshida, as they both deal with images obtained from a 
plurality of cameras. 

One would have been motivated to do this, as input images from multiple camera 
systems are often asynchronous (Yoshida, column 1, lines 19-21) meaning the methods 
could be applied to a greater variety of camera systems. 

As to claim 8, Endo does not expressly disclose multiplexing images onto a 
signal where the sequence of cameras from which the images are coming is not 
predetermined. Yoshida discloses multiplexing images onto a signal (Figure 1,11) and 
further discloses that these images are in an asynchronous state (Column 6, lines 66- 
67) and does not state that the sequence of cameras is known. In addition to this, an 
identification signal is used to identify which camera an image came from (column 3, 
lines 13-16). These pieces of information together, further imply, that the sequence of 
cameras is unknown. 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art to use the image processing techniques in Endo to process 
images that are input from a signal containing images from a plurality of cameras, of 
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unknown sequence, as in Yoshida, as they both deal with images obtained from a 
plurality of cameras. 

One would have been motivated to do this, as it would lower the price of the 
device by reducing the memory requirements (Yoshida, column 1, lines 44-46). 

As to claim 9, Endo as applied above does not expressly disclose that the signals 
on the input signal are synchronous. Yoshida discloses images on input signals being 
synchronous (column 6, line 67 to column 7, line 2). 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill In the art to use the image processing techniques in Endo to process the 
synchronous images in Yoshida, as they both deal with images obtained from a plurality 
of cameras. 

One would have been motivated to do this, as it would reduce the inconvenience 
of dealing with asynchronous signals (Yoshida column 1, lines 20-23), making 
processing easier. 

Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Foote in 
view of Kuno (U.S. Patent 5,243,418). 

Foote as applied above discloses outputting the input image, the step of 
outputting associating the input image with the one state (Figures 30 and 34 and 
Column 23, lines 26-28 and Column 35, lines 41-46). 
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Foote as applied above does not expressly disclose determining if an event is . 
occurring on the input image, the step of determining comparing previous images 
associated with the one state with a present image. 

Kuno discloses determining if an event is occurring on the input image, the step 
of detemiining comparing previous images associated with the one state with a present 
image (column 6, lines 60-67). 

It would have been obvious to a person of ordinary skill in the art, at the time the 
invention was made, to determine if an event was occurring in an image as is done in 
Kuno, when processing image in Foote, as they both involve processing video image 
frames. Doing so, would enable the classifying methods to be used more effectively in 
applications like classifying security system video feeds (i.e. group images showing an 
intrusion together for easy access by security personnel), thus providing motivation. 



Allowable Subject Matter 

Claims 3, and 4 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent fomi including all of the limitations of 
the base claim and any intervening claims. 

As to claim 3, the limitation regarding determining if at least one state exists is 
not expressly disclosed in the prior art. 
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As to Claim 4, the limitation regarding adding a new state that corresponds to the 
input image when the input image information doesn't match any state, is not expressly 
disclosed in the prior art. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ashutosh Upreti whose telephone number is (571) 272- 
7428. The examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (571) 272-7414. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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